az ml endpoint realtime
Note
This reference is part of the azure-cli-ml extension for the Azure CLI (version 2.0.28 or higher). The extension will automatically install the first time you run an az ml endpoint realtime command. Learn more about extensions.
Manage operationalized realtime endpoints.
Commands
| Name | Description | Type | Status |
|---|---|---|---|
| az ml endpoint realtime create-version |
Create a version for realtime endpoint in the workspace. |
Extension | GA |
| az ml endpoint realtime delete |
Delete a realtime endpoint and its version from the workspace. |
Extension | GA |
| az ml endpoint realtime delete-version |
Delete a version for realtime endpoint in the workspace. |
Extension | GA |
| az ml endpoint realtime get-access-token |
Get a token to issue requests a realtime endpoint. |
Extension | GA |
| az ml endpoint realtime get-keys |
Get keys to issue requests against a realtime endpoint. |
Extension | GA |
| az ml endpoint realtime get-logs |
Get logs for a realtime endpoint. |
Extension | GA |
| az ml endpoint realtime list |
List realtime endpoints in the workspace. |
Extension | GA |
| az ml endpoint realtime regen-key |
Regenerate keys for a realtime endpoint. |
Extension | GA |
| az ml endpoint realtime run |
Run a realtime endpoint in the workspace. |
Extension | GA |
| az ml endpoint realtime show |
Show details for a realtime endpoint in the workspace. |
Extension | GA |
| az ml endpoint realtime update |
Update a realtime endpoint in the workspace. |
Extension | GA |
| az ml endpoint realtime update-version |
Update a version for realtime endpoint in the workspace. |
Extension | GA |
az ml endpoint realtime create-version
Create a version for realtime endpoint in the workspace.
az ml endpoint realtime create-version --name
--version-name
[--add-property]
[--add-tag]
[--ar]
[--as]
[--at]
[--autoscale-max-replicas]
[--autoscale-min-replicas]
[--cc]
[--ccl]
[--cf]
[--collect-model-data]
[--cvt]
[--dc]
[--description]
[--ed]
[--entry-script]
[--environment-name]
[--environment-version]
[--failure-threshold]
[--gb]
[--gbl]
[--gc]
[--ic]
[--id]
[--is-default]
[--max-request-wait-time]
[--model]
[--model-metadata-file]
[--no-wait]
[--nr]
[--path]
[--period-seconds]
[--replica-max-concurrent-requests]
[--resource-group]
[--scoring-timeout-ms]
[--sd]
[--st]
[--subscription-id]
[--timeout-seconds]
[--tp]
[--workspace-name]
[-v]Required Parameters
The endpoint name.
The version name to create in an Endpoint.
Optional Parameters
Key/value property to add (e.g. key=value ). Multiple properties can be specified with multiple --add-property options.
Key/value tag to add (e.g. key=value ). Multiple tags can be specified with multiple --add-tag options.
How often the autoscaler should attempt to scale this Webservice. Defaults to 1.
Whether or not to enable autoscaling for this Webservice. Defaults to True if num_replicas is None.
The target utilization (in percent out of 100) the autoscaler should attempt to maintain for this Webservice. Defaults to 70.
The maximum number of containers to use when autoscaling this Webservice. Defaults to 10.
The minimum number of containers to use when autoscaling this Webservice. Defaults to 1.
The number of cpu cores to allocate for this Webservice. Can be a decimal. Defaults to 0.1.
The max number of CPU cores this Webservice is allowed to use. Can be a decimal.
Path to local file containing a conda environment definition to use for the image.
Whether or not to enable model data collection for this Webservice. Defaults to False.
Whether or not this is the control version in an Endpoint. Defaults to False.
Path to a JSON file containing deployment metadata.
Description of the service.
Directory for Azure Machine Learning Environment for deployment. It is the same directory path as provided in 'az ml environment scaffold' command.
Path to local file that contains the code to run for service (relative path from source_directory if one is provided).
Name of Azure Machine Learning Environment for deployment.
Version of an existing Azure Machine Learning Environment for deployment.
When a Pod starts and the liveness probe fails, Kubernetes will try --failure-threshold times before giving up. Defaults to 3. Minimum value is 1.
The amount of memory (in GB) to allocate for this Webservice. Can be a decimal.
The max amount of memory (in GB) this Webservice is allowed to use. Can be a decimal.
The number of gpu cores to allocate for this Webservice. Default is 1.
Path to a JSON or YAML file containing inference configuration.
Number of seconds after the container has started before liveness probes are initiated. Defaults to 310.
Whether or not this is the default version in an Endpoint. Defaults to False.
The maximum amount of time a request will stay in the queue (in milliseconds) before returning a 503 error. Defaults to 500.
The ID of the model to be deployed. Multiple models can be specified with additional -m arguments. Models need to be registered first.
Path to a JSON file containing model registration metadata. Multiple models can be provided using multiple -f parameters.
Flag to not wait for asynchronous calls.
The number of containers to allocate for this Webservice. No default, if this parameter is not set then the autoscaler is enabled by default.
Path to a project folder. Default: current directory.
How often (in seconds) to perform the liveness probe. Default to 10 seconds. Minimum value is 1.
The number of maximum concurrent requests per node to allow for this Webservice. Defaults to 1.
Resource group corresponding to the provided workspace.
A timeout to enforce for scoring calls to this Webservice. Defaults to 60000.
Path to folders that contain all files to create the image.
Minimum consecutive successes for the liveness probe to be considered successful after having failed. Defaults to 1. Minimum value is 1.
Specifies the subscription Id.
Number of seconds after which the liveness probe times out. Defaults to 2 second. Minimum value is 1.
The traffic percentile this version takes in Endpoint.
Name of the workspace containing the service to update.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime delete
Delete a realtime endpoint and its version from the workspace.
az ml endpoint realtime delete --name
[--path]
[--resource-group]
[--subscription-id]
[--workspace-name]
[-v]Required Parameters
The endpoint name to delete.
Optional Parameters
Path to a project folder. Default: current directory.
Resource group corresponding to the provided workspace.
Specifies the subscription Id.
Name of the workspace containing the endpoint to delete.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime delete-version
Delete a version for realtime endpoint in the workspace.
az ml endpoint realtime delete-version --name
--version-name
[--no-wait]
[--path]
[--resource-group]
[--subscription-id]
[--workspace-name]
[-v]Required Parameters
The endpoint name.
The version name to delete.
Optional Parameters
Flag to not wait for asynchronous calls.
Path to a project folder. Default: current directory.
Resource group corresponding to the provided workspace.
Specifies the subscription Id.
Name of the workspace containing the endpoint to delete.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime get-access-token
Get a token to issue requests a realtime endpoint.
az ml endpoint realtime get-access-token --name
[--path]
[--resource-group]
[--subscription-id]
[--workspace-name]
[-v]Required Parameters
Endpoint Name.
Optional Parameters
Path to a project folder. Default: current directory.
Resource group corresponding to the provided workspace.
Specifies the subscription Id.
Name of the workspace containing the endpoint to show.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime get-keys
Get keys to issue requests against a realtime endpoint.
az ml endpoint realtime get-keys --name
[--path]
[--resource-group]
[--subscription-id]
[--workspace-name]
[-v]Required Parameters
Endpoint Name.
Optional Parameters
Path to a project folder. Default: current directory.
Resource group corresponding to the provided workspace.
Specifies the subscription Id.
Name of the workspace containing the endpoint to show.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime get-logs
Get logs for a realtime endpoint.
az ml endpoint realtime get-logs --name
[--init]
[--num_lines]
[--path]
[--resource-group]
[--subscription-id]
[--workspace-name]
[-v]Required Parameters
Endpoint Name.
Optional Parameters
Get logs of init container instead of the scoring container.
Number of log lines to return from tail (default is 5000).
Path to a project folder. Default: current directory.
Resource group corresponding to the provided workspace.
Specifies the subscription Id.
Name of the workspace containing the endpoint to show.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime list
List realtime endpoints in the workspace.
az ml endpoint realtime list [--compute-type]
[--model-id]
[--model-name]
[--path]
[--property]
[--resource-group]
[--subscription-id]
[--tag]
[--workspace-name]
[-v]Optional Parameters
If provided, will only show services that have the specified compute type. (Options are 'ACI', 'AKS', 'AKSENDPOINT').
If provided, will only show services that have the specified model id.
If provided, will only show services that have the specified model name.
Path to a project folder. Default: current directory.
If provided, will filter based on the provided key/value (e.g. key, or key=value). Multiple properties can be specified with multiple --property options.
Resource group corresponding to the provided workspace.
Specifies the subscription Id.
If provided, will filter based on the provided key/value (e.g. key, or key=value). Multiple tags can be specified with multiple --tag options.
Name of the workspace containing the endpoints to list.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime regen-key
Regenerate keys for a realtime endpoint.
az ml endpoint realtime regen-key --key
--name
[--path]
[--resource-group]
[--set-key]
[--subscription-id]
[--workspace-name]
[-v]Required Parameters
Which key to regenerate, if regen is specified. Options: Primary, Secondary.
Endpoint Name.
Optional Parameters
Path to a project folder. Default: current directory.
Resource group corresponding to the provided workspace.
Provide auth value for the specified key.
Specifies the subscription Id.
Name of the workspace containing the endpoint to show.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime run
Run a realtime endpoint in the workspace.
az ml endpoint realtime run --name
[--input-data]
[--path]
[--resource-group]
[--subscription-id]
[--workspace-name]
[-v]Required Parameters
The endpoint name to score against.
Optional Parameters
The data to use for calling the endpoint.
Path to a project folder. Default: current directory.
Resource group corresponding to the provided workspace.
Specifies the subscription Id.
Name of the workspace containing the endpoint to run.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime show
Show details for a realtime endpoint in the workspace.
az ml endpoint realtime show --name
[--path]
[--resource-group]
[--subscription-id]
[--workspace-name]
[-v]Required Parameters
Name of the endpoint to show.
Optional Parameters
Path to a project folder. Default: current directory.
Resource group corresponding to the provided workspace.
Specifies the subscription Id.
Name of the workspace containing the endpoint to show.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime update
Update a realtime endpoint in the workspace.
az ml endpoint realtime update --name
[--add-property]
[--add-tag]
[--ae]
[--ai]
[--description]
[--no-wait]
[--path]
[--remove-tag]
[--resource-group]
[--subscription-id]
[--token-auth-enabled]
[--workspace-name]
[-v]Required Parameters
The endpoint name to update.
Optional Parameters
Key/value property to add (e.g. key=value ). Multiple properties can be specified with multiple --add-property options.
Key/value tag to add (e.g. key=value ). Multiple tags can be specified with multiple --add-tag options.
Whether or not to enable key auth for this Endpoint. Defaults to False.
Whether or not to enable AppInsights for this Endpoint. Defaults to False.
Description of the endpoint.
Flag to not wait for asynchronous calls.
Path to a project folder. Default: current directory.
Key of tag to remove. Multiple tags can be specified with multiple --remove-tag options.
Resource group corresponding to the provided workspace.
Specifies the subscription Id.
Whether or not to enable token auth for this Endpoint. Defaults to False.
Name of the workspace containing the Endpoint to update.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml endpoint realtime update-version
Update a version for realtime endpoint in the workspace.
az ml endpoint realtime update-version --name
--version-name
[--add-property]
[--add-tag]
[--ar]
[--as]
[--at]
[--autoscale-max-replicas]
[--autoscale-min-replicas]
[--cc]
[--ccl]
[--cf]
[--collect-model-data]
[--cvt]
[--dc]
[--description]
[--ed]
[--entry-script]
[--environment-name]
[--environment-version]
[--failure-threshold]
[--gb]
[--gbl]
[--gc]
[--ic]
[--id]
[--is-default]
[--max-request-wait-time]
[--model]
[--model-metadata-file]
[--no-wait]
[--nr]
[--path]
[--period-seconds]
[--replica-max-concurrent-requests]
[--resource-group]
[--scoring-timeout-ms]
[--sd]
[--st]
[--subscription-id]
[--timeout-seconds]
[--tp]
[--workspace-name]
[-v]Required Parameters
The endpoint name.
The version name to create in an Endpoint.
Optional Parameters
Key/value property to add (e.g. key=value ). Multiple properties can be specified with multiple --add-property options.
Key/value tag to add (e.g. key=value ). Multiple tags can be specified with multiple --add-tag options.
How often the autoscaler should attempt to scale this Webservice. Defaults to 1.
Whether or not to enable autoscaling for this Webservice. Defaults to True if num_replicas is None.
The target utilization (in percent out of 100) the autoscaler should attempt to maintain for this Webservice. Defaults to 70.
The maximum number of containers to use when autoscaling this Webservice. Defaults to 10.
The minimum number of containers to use when autoscaling this Webservice. Defaults to 1.
The number of cpu cores to allocate for this Webservice. Can be a decimal. Defaults to 0.1.
The max number of CPU cores this Webservice is allowed to use. Can be a decimal.
Path to local file containing a conda environment definition to use for the image.
Whether or not to enable model data collection for this Webservice. Defaults to False.
Whether or not this is the control version in an Endpoint. Defaults to False.
Path to a JSON file containing deployment metadata.
Description of the service.
Directory for Azure Machine Learning Environment for deployment. It is the same directory path as provided in 'az ml environment scaffold' command.
Path to local file that contains the code to run for service (relative path from source_directory if one is provided).
Name of Azure Machine Learning Environment for deployment.
Version of an existing Azure Machine Learning Environment for deployment.
When a Pod starts and the liveness probe fails, Kubernetes will try --failure-threshold times before giving up. Defaults to 3. Minimum value is 1.
The amount of memory (in GB) to allocate for this Webservice. Can be a decimal.
The max amount of memory (in GB) this Webservice is allowed to use. Can be a decimal.
The number of gpu cores to allocate for this Webservice. Default is 1.
Path to a JSON or YAML file containing inference configuration.
Number of seconds after the container has started before liveness probes are initiated. Defaults to 310.
Whether or not this is the default version in an Endpoint. Defaults to False.
The maximum amount of time a request will stay in the queue (in milliseconds) before returning a 503 error. Defaults to 500.
The ID of the model to be deployed. Multiple models can be specified with additional -m arguments. Models need to be registered first.
Path to a JSON file containing model registration metadata. Multiple models can be provided using multiple -f parameters.
Flag to not wait for asynchronous calls.
The number of containers to allocate for this Webservice. No default, if this parameter is not set then the autoscaler is enabled by default.
Path to a project folder. Default: current directory.
How often (in seconds) to perform the liveness probe. Default to 10 seconds. Minimum value is 1.
The number of maximum concurrent requests per node to allow for this Webservice. Defaults to 1.
Resource group corresponding to the provided workspace.
A timeout to enforce for scoring calls to this Webservice. Defaults to 60000.
Path to folders that contain all files to create the image.
Minimum consecutive successes for the liveness probe to be considered successful after having failed. Defaults to 1. Minimum value is 1.
Specifies the subscription Id.
Number of seconds after which the liveness probe times out. Defaults to 2 second. Minimum value is 1.
The traffic percentile this version takes in Endpoint.
Name of the workspace containing the service to update.
Verbosity flag.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
Feedback
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
Submit and view feedback for